Design of scalable parallel multimedia servers in heterogeneous environments

In the framework of the research activity performed by the Gruppo di Ricerca sull'Informatica Distribuita (GRID group), the authors are currently investigating efficient mechanisms and proper design guidelines to support the development of parallel multimedia servers in heterogeneous systems. The reference architecture for our investigations is based on clusters of workstations, interconnected by high-performance networks. In order to maximize the generality of the devised criteria, however, we base our system on standard programming interfaces (APIs). Where no such interfaces exist, we are working on the definition of general APIs, to make easier the porting effort towards different architectures. According to the principle of not binding the system design to a particular architecture, communication among the components of our parallel servers is carried out through the standard communication library Message Passing Interface (MPI). At present, a number of implementations of MPI for different operating systems and different architectures (Cray T3D, IBM SP2, clusters of workstations) are available. In particular, we have developed a port of a public-domain implementation of MPI, named MPICH, for clusters of workstations on a Myrinet network. This implementation of MPI is based on the user-level high-performance communication library Fast Messages (FM), developed by the CSAG group at the University of Illinois at Urbana-Champaign. The porting effort of MPI on FM is not only motivated by the high performance achievable by FM over Myrinet, but it is also supported by an improvement of some of the features of FM. In particular, we have enriched the credit-based flow-control mechanism of FM, by defining an on-demand credit assignment policy, which lets the applications get a communication throughput near its peak value, even for a large number of communicating processes. Previous work on FM enabled us to introduce some monitoring facilities in the communication library, at the expense of a reasonably low overhead.